Skip to content

Implement CopyToAsync in the FileBufferingReadStream #24499

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 4 commits into from
Aug 3, 2020

Conversation

davidfowl
Copy link
Member

@davidfowl davidfowl commented Aug 1, 2020

  • override Span and Memory overloads and implement array overloads in terms of those overloads.
  • Implemented CopyToAsync (but not CopyTo)
  • Added tests

Fixes #24032

- overrride Span and Memory overloads and implement array overloads in terms of those overloads.
- Implemented CopyToAsync (but not CopyTo)
- Added tests

Fixes #24032
@davidfowl davidfowl requested a review from jkotalik as a code owner August 1, 2020 03:12
@ghost ghost added the area-servers label Aug 1, 2020
@@ -497,8 +497,8 @@ public async Task ReadAsync_DoesNotDisposeBufferedReadStream()
var content = "{\"name\": \"Test\"}";
var contentBytes = Encoding.UTF8.GetBytes(content);
var httpContext = GetHttpContext(contentBytes);
var testBufferedReadStream = new Mock<FileBufferingReadStream>(httpContext.Request.Body, 1024) { CallBase = true };
httpContext.Request.Body = testBufferedReadStream.Object;
var testBufferedReadStream = new VerifyDisposeFileBufferingReadStream(httpContext.Request.Body, 1024);
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moq doesn't support Span.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Inline classes would be so nice!

- Added more tests as this was a missing scenario
{
ThrowIfDisposed();

if (_buffer.Position < _buffer.Length || _completelyBuffered)
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic isn't 100% right, we should be looping in here to fill buffer as much as we can before returning. That isn't a strict contract of Stream though...

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I wait for you to do this before reviewing, or do you want to merge this as is for now?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not making this change in the PR since it isn't a regression.

async Task CopyToAsyncImpl()
{
// At least a 4K buffer
byte[] buffer = _bytePool.Rent(Math.Min(bufferSize, 4096));
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
byte[] buffer = _bytePool.Rent(Math.Min(bufferSize, 4096));
var buffer = _bytePool.Rent(Math.Min(bufferSize, 4096));

{
while (true)
{
int bytesRead = await ReadAsync(buffer, cancellationToken);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
int bytesRead = await ReadAsync(buffer, cancellationToken);
var bytesRead = await ReadAsync(buffer, cancellationToken);

@@ -208,39 +208,41 @@ private Stream CreateTempFile()
FileOptions.Asynchronous | FileOptions.DeleteOnClose | FileOptions.SequentialScan);
}

public override int Read(byte[] buffer, int offset, int count)
public override int Read(Span<byte> buffer)
Copy link
Member

@halter73 halter73 Aug 3, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we shouldn't do something like the runtime does with its Sync/AsyncReadWriteAdapter to avoid duplicating this somewhat complicated logic. Seems like that way we'd be a lot less likely to introduce bugs in just one version of Read(Async).

Copy link
Member

@halter73 halter73 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but you'll get extra kudos from me if you do the Sync/AsyncReadWriteAdapter thing.

@davidfowl
Copy link
Member Author

LGTM, but you'll get extra kudos from me if you do the Sync/AsyncReadWriteAdapter thing.

I already have your kudos 😄

@halter73
Copy link
Member

halter73 commented Aug 3, 2020

Yes, but don't you want extra kudos 😆

@Tratcher
Copy link
Member

Tratcher commented Aug 3, 2020

Yes, but don't you want extra kudos 😆

Try brownies next.

@davidfowl davidfowl added this to the 5.0.0-rc1 milestone Aug 3, 2020
@davidfowl davidfowl merged commit 1f5149a into master Aug 3, 2020
@davidfowl davidfowl deleted the davidfowl/filebuffering branch August 3, 2020 21:14
@@ -349,6 +361,39 @@ public override void Flush()
throw new NotSupportedException();
}

public override Task CopyToAsync(Stream destination, int bufferSize, CancellationToken cancellationToken)
Copy link
Member

@Tratcher Tratcher Aug 5, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How sure are you that you fixed the CopyToAsync issue from #24032? The issue there was that the bufferSize parameter was being set to 1 by the non-virtual CopyToAsync(Stream) if the current stream length was 0 (nothing buffered yet). You're still using the given bufferSize.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oops

@amcasey amcasey added area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions and removed area-runtime labels Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-networking Includes servers, yarp, json patch, bedrock, websockets, http client factory, and http abstractions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FileBufferingReadStream CopyToAsync Performance problem.
6 participants